firing time
Supplementary Material for ' Causality Preserving Chaotic Transformation and Classification using Neurochaos Learning '
This is the supplementary information pertaining to the main manuscript. In this supplementary material, we provide the comparative performance of Neurochaos Learning with Deep Neural Network, 1DConvolutional Neural Network (1D CNN), and Long Short term Memory (LSTM) for evaluation of cause-effect classification of timeseries data generated from coupled chaotic master-slave system and autoregressive (AR) processes. We also check whether each of these architectures are able to preserve cause-effect relationship between the corresponding features extracted from the original cause and effect time series. To evaluate the efficacy of Neurochaos Learning (NL: ChaosNet) and deep learning algorithms for the classification of cause-effect, we used simulated datasets from (a) coupled autoregressive (AR) processes, and (b) coupled 1D chaotic skew tent-maps in master-slave configuration. The governing equations for the coupled AR processes are the following: M(t)=a1M(t 1)+ฮณr(t), (1) S(t)=a2S(t 1)+ฮทM(t 1)+ฮณr(t), (2) where M(t) and S(t) are the independent and the dependent (or the cause and effect) time series respectively; a1 = 0.8, a2 = 0.9, the noise intensity ฮณ = 0.03 and r(t) is independent and identically distributed additive Gaussian noise drawn from a standard normal distribution.
Causality Preserving Chaotic Transformation and Classification using Neurochaos Learning
Discovering cause and effect variables from observational data is an important but challenging problem in science and engineering. In this work, a recently proposed brain inspired learning algorithm namely-Neurochaos Learning (NL) is used for the classification of cause and effect time series generated using coupled autoregressive processes, coupled 1D chaotic skew tent maps, coupled 1D chaotic logistic maps and a real-world prey-predator system. In the case of coupled skew tent maps, the proposed method consistently outperforms a five layer Deep Neural Network (DNN) and Long Short Term Memory (LSTM) architecture for unidirectional coupling coefficient values ranging from 0.1 to 0.7. Further, we investigate the preservation of causality in the feature extracted space of NL using Granger Causality for coupled autoregressive processes and Compression-Complexity Causality for coupled chaotic systems and real-world prey-predator dataset. Unlike DNN, LSTM and 1DConvolutional Neural Network, it is found that NL preserves the inherent causal structures present in the input timeseries data. These findings are promising for the theory and applications of causal machine learning and open up the possibility to explore the potential of NL for more sophisticated causal learning tasks.
c4e5f4de1b3cfc838eec6484d0b85378-Supplemental-Conference.pdf
During training, RNN-based learning algorithms are forced to unfold through the time axis, as explainedinFigure1andSection2. Asaresult,thecorresponding numberofoperations isatleast O(TMN), where Tisthe total time steps and M, N are the number of input and output neurons. On the other hand, event-based learning algorithms only have to deal with cases where a certain neuron fires aspike, and record the relevant information.
Efficient Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoders
Neuman, A. Martina, Petersen, Philipp Christian
Deep learning [6, 29] is a technology that has revolutionized many areas of modern life. The term describes the gradient-based training of deep neural networks. Since its breakthrough in image classification in 2012 [28], deep learning is essentially the only viable technology for this application. Moreover, it is the basis of multiple recent breakthroughs in science [25] and even mathematical research [14]. Recently, deep learning has received wide public attention through the advent of generative AI in the form of large language models such as ChatGPT [39]. It is well-documented that deep learning in modern applications can have extreme requirements on computational resources and the hardware requirements scale in an unsustainable way [52]. In constrained settings, this can become a serious bottleneck preventing the employment of deep learning methods. In addition, these comprehensive computations come with an immense environmental cost.
A Novel Chaos Theory Inspired Neuronal Architecture
B, Harikrishnan N, Nagaraj, Nithin
The practical success of widely used machine learning (ML) and deep learning (DL) algorithms in Artificial Intelligence (AI) community owes to availability of large datasets for training and huge computational resources. Despite the enormous practical success of AI, these algorithms are only loosely inspired from the biological brain and do not mimic any of the fundamental properties of neurons in the brain, one such property being the chaotic firing of biological neurons. This motivates us to develop a novel neuronal architecture where the individual neurons are intrinsically chaotic in nature. By making use of the topological transitivity property of chaos, our neuronal network is able to perform classification tasks with very less number of training samples. For the MNIST dataset, with as low as $0.1 \%$ of the total training data, our method outperforms ML and matches DL in classification accuracy for up to $7$ training samples/class. For the Iris dataset, our accuracy is comparable with ML algorithms, and even with just two training samples/class, we report an accuracy as high as $95.8 \%$. This work highlights the effectiveness of chaos and its properties for learning and paves the way for chaos-inspired neuronal architectures by closely mimicking the chaotic nature of neurons in the brain.